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In  cooperation  with  various  segments  of  the  fruit  industry,  the 
Department  of  Agriculture  contracted  in  19u9  with  the  Industrial  Surveys 
Company  for  monthly  data  that  would  show  consumer  purchases  of  specified 
fruit  and  fruit  products.  The  data  were  to  include  information  showing 
amounts  bought,  prices  paid,  average  size  of  purchase,  and  percentage  of 
households  purchasing.  Quarterly  tabulations  of  these  data  by  regions 
and  types  of  outlet  are  released  in  separate  reports.  Less  frequently, 
reports  are  issued  which  present  buying  practices  by  various  family 
characteristics. 

Since  19u2  the  Industrial  Surveys  Company  has  maintained  a  national 
panel  of  households  based  upon  quota  sampling,  and  during  that  time  has 
conducted  a  number  of  small  scale  studies  on  various  operational  aspects 
of  maintaining  a  consumer  panel,  including  length  and  type  of  the 
.reporting  forms  and  level  of  compensation  needed  to  keep  households 
reporting.  A  study  of  the  possibilities  of  using  a  more  objective  sam- 
pling procedure  was  needed.  Hence,  in  cooperation  with  the  Industrial 
Surveys  Company  and  as  a  part  of  the  contract  with  the  company,  a  study 
was  begun  of  the  possibilities  of  getting  a  random  sample  of  households 
to  keep  records  of  purchases  of  selected  items  and  to  forward  a  report  to 
the  company  at  the  end  of  each  week. 

This  bulletin  presents  the  results  of  an  attempt  to  obtain  the 
cooperation  of  a  probability  sample  of  5U6  households  in  the  New  York 
metropolitan  area,  which  is  recognized  as  one  of  the  most  difficult  areas 
in  the  United  States  with  respect  to  obtaining  a  high  rate  of  response  in 
surveys.  The  field  work  was  started  in  February  1950.  Most  of  the 
effort  to  recruit  households  was  made  during  the  following  3  months, 
although  some  recruiting  work  was  carried  over  until  fall  as  summer  is 
a  very  poor  time  to  recruit  for  a  panel.  Of  the  51*6  households  selected, 
310,  or  60  percent,  agreed  to  cooperate  and  sent  in  one  or  more  weekly 
reports.  During  February  195l,  193  of  the  310  (or  35  percent  of  the  51*6) 
were  still  reporting  regularly,  98  had  dropped  out,  and  19  had  moved  to 
other  dwellings. 

Early  in  1951,  field  work  was  started  on  recruiting  a  national  sample, 
outside  the  New  York  metropolitan  area,  of  approximately  1,100  households 
specified  by  probability  sampling.  The  sample  of  1,100  households  was  a 
subsample  of  a  larger  sample  which  was  expected  to  be  set  up  eventually. 
Although  the  effort  to  recruit  each  household  in  the  national  sample  was 
less  than  that  for  the  New  York  area,  the  initial  rate  of  cooperation  was 
nearly  70  percent  as  compared  to  60  percent  in  the  New  York  area.  A  re- 
port on  this  project  is  to  be  published  later. 

Earl  E.  Houseman,  statistician 
Bureau  of  Agricultural  Economics 
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PROBLEMS  OF  ESTABLISHING  A  CONSUMER  PANEL  IN  THE 
NEW  YORK  METROPOLITAN  AREA 

Prepared  by  Industrial  Surveys  Company 
under  contract  as  authorized  by 
the  Research  and  Marketing  Act  of  19U6  (Title  II) 
Edited  for  publication  by  Bureau  of  Agricultural  Economics 

INTRODUCTION  AND  STATEMENT  OF  OBJECTIVES 

Although  a  probability  sample  of  households  may  be  specified  for  a 
survey,  the  desired  information  is  usually  not  obtained  from  all  of  the 
households  designated  by  the  design.  The  rate  of  response,  which  varies 
widely  from  one  survey  to  another,  is  conditioned  by  numerous  factors, 
chief  of  which  are  the  nature  of  the  survey  and  the  effort  put  forth  to 
obtain  the  required  information  from  potential  respondents.  The  survey 
discussed  here  differs  from  the  usual  survey  in  that  cooperation  of  a 
sample  household  or  panel  is  required  for  an  extended  period  of  time. 

For  purposes  of  the  study  upon  which  this  report  is  based,  a  "con- 
tinuous consumer  panel"  or  "panel"  refers  to  a  sample  of  households 
reporting  by  mail  each  week  their  day-to-day  purchases  of  certain  food 
and  personal-care  items.  Because  the  bulk  of  a  panel  remains  constant 
throughout  5  given  year,  and  substantially  so  from  year  to  year,  the  pur- 
chases of  a  given  household  at  one  point  in  time  may  be  linked  to  the 
same  household's  purchasing  behavior  at  a  later  point  in  time.  The 
printed  form  used  by  sample  households  to  record  daily  their  purchases  of 
food  and  personal-care  items  is  referred  to  in  this  report  as  a  "diary." 
For  each  item,bought  the  household  was  requested  to  provide  information 
as  to  brand,-' quantity,  size,  and  outlet  in  which  bought.  In  the  main, 
the  information  requested  was  on  packaged  foods.  Thus  high-velocity 
items  such  as  railk,  meat,  bread,  and  fresh  vegetables  were  excluded. 
The  average  number  of  entries  made  per  week  was  about  17.  It  is  esti- 
mated that  the  recording  of  such  items  took  10  to  30  minutes  a  week. 

The  over-all  objective  of  the  project  was  to  study  the  problems 
associated  with  the  use  of  a  probability  sample  design  to  establish  and 
maintain  a  continuous  consumer  panel.  By  a  probability  sample  design  is 
meant  one  in  which  each  member  of  the  universe  can  be  assigned  a  known 
nonzero  chance  of  being  selected  for  inclusion  in  the  sample.  Note  that 
the  word  "sample"  refers  to  all  households  selected  and  not  the  partic- 
ular households  that  cooperate. 


1/  Information  as  to  brand  was  for  clients  other  than  the  Department 
of  Agriculture. 
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The  specific  objectives  of  the  project  were:  (1)  To  ascertain  the 
degree  to  which  the  cooperation  of  a  probability  sample  of  households 
in  which  continuous  records  of  purchases  of  food  and  personal -care  items 
would  be  kept  could  be  obtained;  (2)  to  explore  and,  so  far  as  possible, 
to  overcome  obstacles  to  cooperation;  (3)  to  examine  the  influence  of 
noncooperation  on  the  accuracy  of  results;  and  (U)  to  investigate  proce- 
dures to  be  followed  in  selecting  substitute  households  for  households 
that  refuse  to  join  the  panel. 

DESIGN  OF  THE  SAMPLE 

In  planning  the  study,  consideration  was  given  to  the  possibility 
of  using  an  experimental  plan  such  as  the  factorial  design.?./  Under  this 
scheme  it  would  have  been  possible  to  test  in  various  combinations  such 
factors  as  length  and  form  of  the  diary,  the  level  of  compensation  for 
cooperation,  the  method  of  soliciting  cooperation,  and  the  method  of 
training  households.  The  major  deterrents  to  the  use  of  a  factorial  de- 
sign for  this  project  weres 

(1)  A  rigid,  standardized  recruiting  approach  would  have  been 
necessary  for  all  sample  households  that  received  the  same  treatment 
combination.  Previous  experienced/had  indicated  that  each  household 
presented  a  unique  problem  and  that  a  flexible  recruiting  procedure  which 
could  be  varied  at  will  from  family  to  family  or  from  time  to  time  for  a 
given  family  would  lead  to  a  higher  rate  of  cooperation. 

(2)  The  expense  and  administrative  difficulties  that  arise  from  a 
factorial  design  would  have  been  excessive  because,  as  a  safeguard 
against  confounding  of  interviewer  differences  with  other  factors,  it 
would  have  been  necessary  to  include  interviewers  as  a  factor  under  test. 
That  is,  randomization  of  interviewers  among  households  would  have  been 
necessary,  with  a  consequent  increase  in  travel  expense  and  a  lessening 
of  immediate  administrative  control  over  the  interviewers. 

(3)  Inasmuch  as  considerable  "missing  data"  were  anticipated,  retain- 
ing the  orthogonality  of  the  design  would  have  been  impossible. 

(U)  If  an  elaborate  design  is  used  the  limitation  on  size  of  sample 
is  such  that  the  number  of  households  that  could  receive  any  one  treat- 
ment combination  would  be  too  small  for  purposes  of  making  reliable 
comparisons  among  the  various  treatment  combinations. 

For  these  reasons  it  was  decided  that  more  information  could  probably 
be  obtained  per  dollar  spent  if  use  were  made  of  a  less  rigorous  approach. 
As  the  plan  was  finally  evolved,  interviewers  were  instructed  to  use 
varying  approaches  in  attempting  to  get  the  sample  households  to  cooperate. 


2/  For  details  as  to  the  structure  of  a  factorial  design  see,  for  example, 
Cochran,  W.  G. ,  and  Cox,  G.  M.  Experimental  Designs.  John  Wiley  and 
Sons,  Inc.,  19$0.  Ch.  5. 

3/  A  pretest  of  the  feasibility  of  using  a  probability  sample  for  a 
panel  was  carried  out  in  Chicago  in  19h9   by  the  Industrial  Surveys  Company. 
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Thus  the  recruiting  approach  was  to  be  modified  to  fit  particular  situa- 
tions. The  details  of  this  decision  are  elaborated  later  in  this  report. 
The  structure  of  the  survey  design  used  to  identify  sample  households  is 
discussed  in  succeeding  paragraphs. 

The  standard  metropolitan  areak/of  New  York  was  selected  as  the 
locale  for  the  study  largely  because  of  the  number  and  diversity  of  prob- 
lems associated  with  the  area,  such  as  literacy  of  population,  language 
difficulties,  and  wide  variation  in  living  conditions.   Because  of  this 
diversity,  the  experience  gained  would  be  helpful  if  it  were  decided  to 
attempt  the  recruiting  of  a  national  probability  sample  of  households. 

The  fipal  reporting  sample  was  to  consist  of  approximately  350 
households. dJ   The  number  of  households  in  the  area  was  estimated  to  be 
3,1*20,000.  As  a  cooperation  rate  of  65  percent  was  anticipated,  it  was 
estimated  that  538  ■  (350/.65)  households  should  be  specified  for  inclu- 
sion in  the  sample.  This  specified  sample  size  of  538  corresponded  to  an 
over-all  sampling  rate  of  538/3, U20, 000  or  0.00016.  The  sampling  was 
carried  out  in  three  stages:  (1)  Selection  of  places.?/,  (2)  selection  of 
blocks  within  places,  (3)  selection  of  dwelling  units  within  blocks. 

In  the  first  stage  of  the  sampling,  the  five  boroughs  were  automat- 
ically included  because  of  the  number  of  dwelling  units  in  them.  The 
remaining  places  were  grouped  into  18  strata  of  approximately  equal  size 
on  the  basis  of  number  of  dwelling  units,  geographic  location,  and 
average  rent.  An  additional  stratum  was  comprised  of  large  housing 
developments  since  19l|0.  Within  each  of  the  16  strata  one  place  was 
selected  with  a  probability  proportionate  to  its  191*0  number  of  dwelling 
units.  The  within-place  sampling  rates  were  then  determined  by  the 
relationship  r  =»  R/P  where  r  »  within-place  sampling  rate,  R  ■  over-all 
sampling  rate,  and  P  =  probability  of  a  place  being  selected. 

The  number  of  blocks  chosen  in  each  place  was  set  to  provide  an 
average  of  two  sample  households  per  block.  For  10  of  the  18  sample 
places,  block  statistics  from  the  19h0   Census  were  available  and  were 
used  to  stratify  blocks  by  average  rent  and  geographic  location.  Sample 
blocks  were  selected  with  probabilities  proportionate  to  the  number  of 
dwelling  units  reported  in  the  19l*0  Census.  In  the  places  without  block 
statistics,  all  blocks  were  stratified  geographically  and  sample  blocks 
were  selected  with  equal  probabilities.  Within  the  19th  stratum  which 


k/   Standard  metropolitan  areas  were  defined  by  a  Federal  interagency 
committee  which  was  set  up  late  in  I9I4.7  under  the  direction  of  the 
Bureau  of  the  Budget. 

5/  The  census  definition  of  a  household  was  used.  As  defined  for  census 
purposes,  a  household  consists  of  those  persons  who  live  in  a  dwelling 
unit  which  in  essence  is  a  room  or  group  of  rooms,  occupied  or  intended 
for  occupancy  as  separate  living  quarters,  having  separate  cooking  equip- 
ment. This  excludes  certain  living  quarters  in  dormitories,  transient 
hotels,  tourist  courts,  and  institutions. 

6/  Place  is  used  as  a  general  term  to  include  any  of  the  five  boroughs, 
an  incorporated  city  or  town, or  the  remainder  of  a  minor  civil  division 
after  deletion  of  any  towns  or  cities. 
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was  comprised  of  housing  projects,  each  floor  of  an  apartment  building  was 
regarded  as  a  "block"  and  a  sample  of  floors  was  selected  with  probabili- 
ties proportionate  to  the  estimated  number  of  dwelling  units  on  the  floors. 

A  special  field  staff  listed  all  dwelling  units  in  the  sample  blocks. 
Difficulty  was  encountered  in  gaining  entrance  to  certain  apartment  hotels 
and  high-rental  apartment  houses,  but  with  one  exception,  these  problem 
apartment  houses  were  eventually  enumerated.  Two  blocks  in  very  low 
rental  districts  (so-called  "violation  areas"  where  overcrowding  was 
prevalent)  could  not  be  enumerated  because  of  lack  of  cooperation  on  the 
part  of  both  landlords  and  tenants.  Substitutes  for  these  two  blocks 
were  selected  from  within  the  same  low-rental  districts.  The  actual 
selection  of  sample  dwelling  units  was  carried  out  in  the  field  by  the 
enumerative  staff  in  order  to  identify  the  names  of  the  householders  to 
be  contacted.  Dwelling  units  within  the  sample  blocks  were  selected  by 
means  of  a  systematic  sampling  from  a  random  starting  point.  The  correct- 
ness of  this  field  operation  was  verified  in  the  central  office.  In  this 
way,  575  dwelling  units  were  identified,  26  of  which  did  not  contain 
eligible  households  and  3  of  which  were  vacant.  Thus,  in  all,  51±6  house- 
holds were  designated  for  the  sample. 

PUN  OF  OPERATION 

Introductory  Statement 

The  operational  goal  of  the  study  was  to  obtain  complete  and  accurate 
continuous  weekly  reports  of  purchases  of  certain  foods  and  personal-care 
items  by  members  of  the  sample  households.  This  information  was  to  be 
obtained  from  each  household  by  means  of  a  self-administering  form  known 
as  a  "diary."  Each  household  agreeing  to  participate  in  the  panel  was  to 
receive  at  the  outset  a  merchandise  premium  worth  about  $5.00.  In 
addition,  each  was  to  be  remunerated  in  "points"  which  could  be  exchanged 
for  merchandise  premiums.  The  number  of  points  to  be  received  depended 
upon  the  number  of  members  in  the  household,  the  number  of  diaries  mailed 
in  during  a  3-month  period,  and  whether  each  weekly  diary  was  mailed  by  a 
certain  dead-line  date.  The  average  monthly  remuneration  was  valued  at 
$2.15  in  merchandise  premiums.!/ 

A  member  of  the  household  (usually  the  housewife)  was  to  be  respon- 
sible for  keeping  the  diary.  This  recorder,  or  monitor,  was  to  be  given 
an  intensive  training  program,  following  which  the  household  would  be 
placed  on  mail  control  -  that  is,  each  household  was  to  receive  four  weekly 
diaries  each  month  and  the  monitor  was  to  mail  back  a  completed  diary  each 
week.  A  household  on  mail  control  which  failed  to  return  a  diary,  was  late 
in  returning  it,  or  did  a  poor  reporting  job  (as  evidenced  by  editing)  was  to 
receive  a  personal  call  from  the  interviewer  who  had  initially  contacted  it. 


7/  In  the  past  the  level  of  compensation  necessary  to  maintain  families 
reporting  on  a  satisfactory  basis  has  undergone  close  scrutiny  by  the 
Industrial  Surveys  Company.  This  level  of  compensation  was  found  to  be 
about  the  minimum  that  would  retain  loyal  reporting  and  prevent  households 
from  discontinuing  cooperation  because  the  level  of  compensation  was  too  low. 


Selection  and  Training  of  Field  Force 

A  list  of  more  than  100  names  of  prospective  interviewers,  all  of 
whom  were  women,  was  compiled  from  the  records  of  the  Industrial  Surveys 
Company  and  from  contacts  with  other  research  organizations.  The  job 
requirements  for  the  full-time  field  staff  were: 

(1)  Age  -  30  to  U5  years. 

(2)  Marital  status  -  married. 

(3)  Family  size  -  no  children  under  10  years  of  age  unless  adequate 
provision  can  be  made  for  their  care  both  during  the  day 

and  evening. 
(U)  Education  -  2  to  It  years  of  college  desirable. 

(5)  Experience  -  responsible  position  working  with  women. 
Interviewing  experience  desirable,  especially  if  on  surveys 
employing  a  probability  sample  design.  No  house-to-house  selling 
and  preferably  no  retail  selling.  Social  work  acceptable. 

( 6)  Appearance  -  neat  personal  appearance,  conservative  dresser. 

(7)  Health  -  ability  to  work  long  hours  and  in  various  types  of 
weather. 

(8)  Personality  -  pleasing.  Must  like  all  kinds  of  people.  Ability 
to  carry  on  a  conversation  with  all  types  of  people.  Not  over- 
bearing or  too  aggressive.  Good  listener.  Persevering  yet 
tolerant. 

(9)  Availability  -  available  to  work  nights,  Saturdays,  and  Sundays. 

On  the  basis  of  the  interviewer's  past  experience,  recommendations 
and  personal  interviews  by  the  person  in  charge  of  field  operations,  all 
except  17  prospective  interviewers  were  eliminated.  Further  interviewing 
of  the  candidates  resulted  in  the  hiring  of  11  for  full-time  work.  No 
attempt  was  made  initially  to  select  interviewers  who  could  handle  special 
language  problems. 

Planning  of  an  interviewer  training  program  necessitated  an  explicit 
formulation  of  the  tasks  required  of  the  successful  interviewer.  These 
were  determined  to  be:   (1)  To  approach  sample  households  and  present  the 
importance  of  panel  membership;  (2)  to  obtain  from  every  sample  household 
certain  classificatory  information  on  such  items  as  income,  occupation  of 
employed  members,  and  education  of  all  members;  (3)  to  train  the  monitor 
in  accurate  reporting  for  the  households  that  agreed  to  join  the  panel; 
(li)  to  visit  at  periodic  intervals  certain  households  that  indicated  that 
they  might  drop  from  the  panel;  (5>)  to  record  each  experience  and  associa- 
tion with  the  household. 

To  provide  each  worker  with  the  confidence,  enthusiasm,  and  skill 
needed  to  carry  out  these  tasks,  a  conference-type  training  program  lasting 
8  days  was  instituted.  During  the  early  part  of  the  training  program, 
interviewers  were  familiarized  with  the  over-all  objectives  and  purposes  of 
the  panel.  Particular  stress  was  placed  on  the  importance  of  securing  the 
cooperation  of  as  many  sample  households  as  possible.  The  nature  of  the 
diary  was  explained  and  interviewers  were  required  to  keep  the  diary  during 
the  training  program  in  order  to  understand  the  problems  involved.  Prob- 
able sources  of  difficulty  in  contacting,  recruiting,  training,  and 


maintaining  the  sample  households  were  emphasized,  and  interviewers  were 
advised  on  alternative  methods  of  coping  with  anticipated  difficulties. 
At  all  times,  they  were  encouraged  to  participate  in  the  discussion  of 
problems  and  to  ask  questions.  Two  days  of  the  training  program  were 
devoted  to  practice  work  in  the  field,  and  the  final  day  was  spent  in 
discussing  the  problems  that  arose  during  the  practice  work. 

Procedures  for  Controlling  and  Recording  Field  Operations 

Before  initial  contact  was  made  by  interviewers,  an  introductory  letter 
describing  the  nature  and  activities  of  the  panel  was  mailed  to  each  sample 
household.  In  order  to  carry  out  the  recruiting  task,  each  interviewer 
was  provided  with  the  following  materials:  (1)  Four  diaries  in  which  the 
household  name,  number,  and  address  were  typed;  (2)  a  merchandise  catalog 
showing  prizes  available  to  panel  participants;  (3)  a  household-recruiting 
manual;  (h)   a  reference  guide  for  panel  households;  (5)  a  recruiting- 
call  report;  (6)  an  interviewer  identification  card. 

The  households  successfully  recruited  were  in  a  training  status  for 
about  3  months,  although  some  households  required  very  little  training. 
Diaries  of  those  who  mailed  them  in  while  in  training  status,  were  edited 
in  the  central  office  and  returned  to  the  interviewers,  who  then  discussed 
with  the  monitors  any  errors  that  were  made.  Certain  households  retained 
the  diaries  for  pick-up  by  the  interviewers,  who,  at  the  time  of  pick-up, 
discussed  with  the  monitor  any  errors  in  these  diaries. 

Once  training  had  been  completed,  no  diaries  were  returned  to  the 
interviewers  for  consultation  with  the  monitors  unless  an  excessive  number 
of  errors  had  been  detected.  In  the  latter  event,  a  special  call  (termed 
a  maintenance  call)  was  arranged.  Maintenance  calls  were  also  made  if  a 
monitor  had  stopped  reporting  or  showed  evidence  of  erratic  reporting. 
In  these  cases,  intensive  efforts  were  made  to  regain  the  cooperation  of 
the  delinquent  households. 

During  recruiting  and  maintenance  operations,  individual  interviewers 
sent  daily  reports  to  supervisory  personnel.  In  addition,  the  field 
supervisor  and  the  person  in  over-all  charge  of  the  project  held  a  weekly 
conference  with  each  interviewer.  At  this  time,  a  review  of  the  status  of 
each  sample  household  was  conducted  and  appropriate  action  was  recommended 
for  the  following  week's  activities. 

To  record  contacts  with  sample  households,  the  following  forms  were 
employed:  (1)  Recruiting  Call  Report  (see  Appendix  A);  (2)  Training  Call 
Report  (see  Appendix  B);  (3)  Special  Action  Form?./;  (k)   Daily  Time  and 
Expense  Report2/;  (5)  Master  Control  Forrai2/. 

8/  This  form  was  designed  to  record  any  extraordinary  measures  taken  by 
the  company  to  secure  or  maintain  household  cooperation.  A  more  explicit 
definition  of  the  operations  encompassed  by  special  action  is  presented 
in  a  later  section  of  this  report. 

9/  All  forms  relevant  to  the  day's  activities  (recruiting,  training,  and 
special -action  forms)  accompanied  this  daily  report. 

10/  This  form  was  designed  to  provide  a  summary  for  each  household  for 
recording  all  types  of  calls  by  purpose  and  a  record  of  diaries  sent  in, 
as  well  as  the  points  earned. 
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These  forms  were  designed  to  obtain  the  necessary  information  for  the 
analysis  phase  of  the  project.  They  did  not  provide  cost  data  on  a  house- 
hold or  call  basis.  Cost  was  to  be  evaluated  in  terms  of  relative  effort 
expended  among  the  various  types  of  operation  -  contacting,  recruiting, 
maintenance.  The  measure  of  effort  was  to  be  expressed  in  terms  of  number 
of  calls  expended  on  the  various  operations,  rather  than  on  a  cost  basis. 

PROGRESS  OF  WORK  IN  THE  FIELD 

On  February  8,  19$0,   the  work  of  recruiting  the  5kb   sample  households 
was  begun.  During  February  and  March  about  65  percent  of  the  sample  house- 
holds received  one  or  more  recruiting  calls.  By  a  recruiting  call  is 
meant  one  in  which  the  interviewer  discusses  panel  membership  with  the 
potential  household  monitor.  Thus,  although  the  potential  monitor  might 
have  been  home,  the  call  was  classified  as  a  not-at-home  recruiting  call 
if  there  was  no  opportunity  to  discuss  panel  membership.  Early  in  April 
the  field  force  was  reduced  from  11  to  7  -  one  interviewer  had  resigned 
and  3  were  released.  By  this  time  the  main  function  of  the  field  force  had 
changed  from  recruiting  to  that  of  training  and  maintaining  the  households 
that  had  been  recruited.  ~±/   Some  recruiting  work  continued,  although  the 
primary  emphasis  was  on  consolidation  of  the  recruiting  gains  already  made. 

During  the  first  part  of  May  the  status  of  all  sample  households  was 
reviewed.  This  review  indicated  that  28  percent  of  the  sample  households 
had  not  yet  been  contacted,  and  efforts  designed  to  reach  these  households 
were  intensified.  Three  new  interviewers,  including  two  bilingual  workers, 
were  hired.  Early  in  June  it  was  decided  to  diminish  recruiting  effort 
until  September  because  of  indications  that  recruiting  efforts  would  meet 
with  little  success  during  the  summer.  A  number  of  households  had  indicated 
reluctance  to  discuss  panel  membership  because  of  pending  summer  plans. 

In  September  the  status  of  each  sample  household  was  again  reviewed. 
By  this  time,  29k   households  had  been  recruited.  A  total  of  170,  comprised 
of  the  following  groups,  were  selected  to  receive  further  calls:  (1)  house- 
holds not  yet  contacted!  (2)  households  that  had  requested  that  the  inter- 
viewer return  at  the  end  of  the  summer;  (3)  households  that  had  been  recruit- 
ed but  had  stopped  reporting  for  a  reason  that  might  be  altered  over  a  period 
of  time;  (k)   households  that  had  refused  but  might  be  persuaded  to  reconsider; 
(5)  households  for  which  complete  classification  data  were  not  available. 

Each  interviewer  was  provided  with  detailed  written  instructions  in 
regard  to  the  type  of  action  to  be  taken  with  these  170  households.  Calls 
were  completed  on  these  households  on  November  25,  195>0,  with  the  following 
results:   (1)  20  percent  were  recruited  or  re-recruited;  (2)  70  percent 
refused  to  join  or  rejoin;  (3)  10  percent  were  not  contacted. 

In  November  plans  were  made  to  select  a  subsample  of  162  noncooperative 
households  for  which  substitutes  were  to  be  selected.  The  actual  proce- 
dures used  and  a  summary  of  results  are  presented  later  in  this  report. 

11/  A  household  was  defined  as  recruited  if  it  had  completed  at  least  one 
diary  after  agreeing  to  join  the  panel.  This  definition  is  followed 
throughout  the  report. 


By  February  195l,  the  project  had  been  under  way  for  a  year.  The 
status  of  the  sample  households  at  that  time  is  presented  in  table  1. 

Table  1.-  Status  of  sample  households,  February  28,  1951 


Households 

Status  of  household 

Total 

s  Percentage  of 
:  total  sample 

Recruited s 

Number 

193 

x              98 

19 

Percent 
35 

Dropped  

Moved  

18 
k 

Total  .... 

310 

11 

Refused  

210 

1U 
12 

38 

Called  on  but  unable  to 

discuss  membership  .  .  . 
Never  called  on  

3 
2 

Total  sample  . 

x            &6 

100 

The  1U  households  called  on  but  unable  to  discuss  panel  membership 
represent  a  number  of  unusual  cases.  To  illustrate,  several  households 
had  recently  suffered  deaths  so  that  no  monitor  existed  and  in  6  cases  the 
household  had  moved  between  visits  by  the  interviewer.  The  12  households 
never  called  on  were  in  apartment  hotels.  All  of  these  households  were 
later  contacted  after  permission  to  enter  the  hotels  was  obtained. 

During  the  course  of  the  project,  19  of  the  sample  households  moved 
to  other  dwelling  units.  The  field  force  was  instructed  to  try  to  recruit 
the  "new"  households  residing  in  the  sample  dwelling  units,  regardless  of 
the  status  of  the  households  previously  residing  there.  However,  this 
report  is  confined  to  a  consideration  of  the  original  sample  households 
that  were  contacted. 

DESCRIPTION  AND  EVALUATION  OF  RECRUITING  EFFORTS  AND  RESULTS 


The  preceding  sections  of  this  report  have  been  concerned  primarily 
with  an  outline  of  the  general  structure  of  the  study.  Some  account  of 
progress  in  the  field  was  given,  but  the  detailed  description  of  methods 
involved  and  the  interpretation  of  results  have  been  deferred  to  this  and 
later  sections  of  the  report.  The  present  section  consists  of  a  report 
on  the  nature  and  extent  of  efforts  made  to  recruit  sample  households. 

The  extent  of  the  recruiting  effort  in  terms  of  number  of  recruiting 
and  not-at-home  recruiting  calls  made  and  number  of  households  recruited 
is  indicated  in  table  2.  For  purposes  of  tabulation  the  relatively  small 
number  of  calls  made  when  the  potential  monitor  was  home  but  unwilling  to 
discuss  panel  membership  was  included  among  the  not-at-home  recruiting 
calls. 
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Table  2.-  Number  of  recruiting  and  not-at-home  recruiting  calls  made 
and  households  recruited  by  number  of  recruiting  calls 


Households  receiving:    Not-at-home 
recruiting  calls  :  recruiting  calls 


Households 
recruited 


Number 

of 

recruit-: 

ing 
calls 


: Percentage 
Total  households 
1/  :receiving 

: stated  number 
:of  calls 


Total 
2/ 


Per 

recruit 

call 


Total  :  Per 
3/  : recruit 
:  call 


Per 

call 


Number    Percent 


Number    Number  Number  Number 


1  ... 

$20 

100 

79U 

1.5 

212         OsUl 

0.16 

2  ... 

212 

kl 

161 

08 

bh           ,30 

.17 

3  ... 

79 

15 

53 

23 

k  ... 

21 

k 

31 

h/  .8 

10    V  .31 

k/    ,17 

5  ... 

6 

1 

5 

1 

6  ... 

2 

- 

- 

- 

Totalo . 

8Uo 

l,0hh 

1.2 

310          .37 

.17 

1/  Data  in  this  column  are  presented  in  terms  of  households  receiving  1 
or  more  recruiting  calls,  2  or  more  recruiting  calls,  etc.  Thus,  212  of 
520  received  2  or  more  recruiting  calls  and  308  or  (512-212)  received  only 
1  recruiting  call.  The  sum  of  the  first  column,  81|0,  is  the  total  number 
of  recruiting  calls  made. 

2/  This  column  shows  the  number  of  not-at-home  calls  made  to  effect  the 
first  recruiting  calls,  the  second  recruiting  calls,  etc. 

3/  This  column  shows  the  number  of  households  recruited  on  the  first 
recruiting  call,  on  the  second  recruiting  call,  etc. 

h/   Average  for  households  recruited  on  third,  fourth,  fifth ,  and  sixth 
recruiting  calls  combined. 


The  most  important  facts  given  by  table  2  are:   (1)  Sixty  percent  or  310 
of  the  520  households  receiving  recruiting  calls  were  recruited;  (2)  the 
average  number  of  households  recruited  per  recruiting  call  dropped  from 
O.hl  for  the  first  recruiting  call  to  0.31  for  later  calls;  (3)  hi   percent 
of  the  sample  households  receiving  recruiting  calls  received  2  or  more; 
(h)   the  average  number  of  not-at-home  calls  per  recruiting  call  diminished 
by  almost  half  following  the  first  recruiting  call.  By  appropriate 
scheduling,  interviewers  increased  the  likelihood  of  completing  a  recruit- 
ing call;  (5)  if  both  recruiting  and  not-at-home  calls  are  considered,  the 
average  number  of  households  recruited  per  call  was  about  the  same  for 
the  first  as  for  later  calls. 


After  each  recruiting  call,  the  number  of  households  that  were  to 
receive  additional  calls  was  determined  by  a  screening  procedure  that 
eliminated  those  households  on  which  further  effort  appeared  to  be  useless. 
Some  of  the  households  scheduled  to  receive  additional  recruiting  calls 
did  not  receive  them,  as  it  was  found  on  a  not-at-home  call  that  further 
effort  would  be  useless.  For  example,  the  monitor's  husband  might  have 
answered  the  call  and  insisted  that  the  interviewer  not  return. 
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It  might  be  anticipated  that  households  that  required  more  than  one 
recruiting  call  before  joining  the  panel  would  be  most  likely  to  dis- 
continue reporting.  Although  table  3  indicates  some  confirmation  of  such 
a  hypothesis,  the  evidence  is  not  so  great  as  to  suggest  that  recruiting 
efforts  beyond  the  first  call  were  not  worth  while,  particularly  in  view 
of  the  fact  that  the  number  of  households  recruited  per  call  (table  2)  was 
about  the  same  on  later  calls  as  on  the  first  call. 

Members  of  the  households  which  refused  to  cooperate  were  much  less 
likely  to  be  found  at  home.  As  shown  in  table  k   approximately  three  times 
as  many  not-at-home  calls  were  required  per  recruiting  call  for  the 
refusal  households  as  for  the  cooperating  households.  In  terms  of  the 
total  recruiting  effort,  table  5,  U6  percent  of  the  recruiting  calls,  or 
386  of  8U0,  and  70  percent  of  the  not-at-home  recruiting  calls  were  made 
on  the  refusal  households,  which  constituted  k0  percent  of  the  sample. 
However,  the  amount  of  effort  spent  on  refusal  households  is  not  a  sound 
criterion  for  deciding  the  number  of  calls  to  make.  That  problem  should 
be  considered  in  terms  of  the  gains  for  marginal  increases  in  the  recruit- 
ing effort. 

Table  3.-  Distribution  of  households  by  status  and  number 
of  recruiting  calls  made 


be 

Households 

of 

recruit- 
ing 
calls 

Reporting 

:    Recruited 
:   but  dropped 

'Reporting 
^as  a  per- 
centage of 
Recruited 

Refusal 

: Percentage 
-Total  :distribu- 
:  tion 

:      t Percent age 
:Total  :distribu- 
s      :  tion 

Percentage 
Total  :distribu- 
:  tion 

Number  Percent 

Number  Percent 

61      63 

22      22 

11      11 

k                 k 

Percent 

71 
66 
52 
60 

Number  Percent 

1  ... 

2  ... 

3  ... 
h   ... 

5  ... 

6  ... 

151      71 

U2      20 

12       6 

6       3 

1 

95            US 
70      33 
31      15 
10      5 
2      l 
2       1 

Total.. 

t  212     100 

98     100 

68 

210     100 

Table  U. 


Number  of  not-at-home  recruiting  calls  by  status 
of  household  and  recruiting  call 


Households 

Number  of 

Reporting 

Dropped 

:      Refusal 

recruiting 
calls 

Total 

:Number  per 
: recruiting 
:   call 

Total 

: Number  per 
: recruiting 
:   call 

:  Total 

:Number  per 
trecruiting 
:   call 

Number 

Number 

1.2 
.2 
.1 

Number 

85 

20 

9 

Number 

1.1* 
.5 
.2 

Number 

326 
2li0 
163 

Number 

1 

2 

3,U,5,  &  6 

179 

13 

9 

3.U 
1.7 
1.1 

Total  ..... 

201 

.7 

llli 

.7 

?29 

1.9 
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Table  5.-  Summary  of  calls  made  by  status  of  household 


Status 

of 

household 


Number 
of 
households 


Number 

of 

recruiting 

calls 


Number  of 
not- at -home 
recruiting 
calls 


Number  of 
recruiting 
calls  per 
household 


:  Number  of 

i  not -at -home 

t  calls  per 

:  recruiting 
:   call 


Reporting. 
Dropped  . . 
Refusal  .. 
Total 


212 


210 


520 


Number 
300 
386 


Number 

201 
llil 
729 


81*0 


Jr2°55I 


Number 

1.6 

1.9 
— 176— 


Number 

0.7 

.7 

1.9 


1.2 


In  an  earlier  section  it  was  stated  that  the  interviewers  were  encour- 
aged to  use  various  approaches  in  attempting  to  recruit  the  sample  house- 
holds. Nevertheless,  these  differential  recruiting  techniques  operated 
within  a  fairly  rigid  frame  of  reference.  One  of  the  following  four  courses 
of  action,  which  will  be  referred  to  as  "special  action",  were  taken  when 
it  appeared  that  a  household  was  unlikely  to  be  recruited  without  such 
action: 

(1)  The  type  and  method  of  compensation  offered  the  household.  In 
some  instances,  for  example,  a  household  was  offered  twice  the 
standard  compensation  for  participating  in  the  panel  when  it 
appeared  that  resistance  to  cooperation  might  be  overcome  by 
additional  compensation. 

(2)  The  interviewer  assigned  to  recruit  the  household.  In  certain 
cases,  when  a  particular  interviewer  was  unsuccessful,  a 
different  interviewer  was  assigned  to  the  uncooperative 
household. 

(3)  The  language  in  which  the  recruiting  process  was  conducted.  In 
certain  non-English  speaking  households,  use  was  made  of  Spanish- 
and  Italian-speaking  interviewers. 

(h)   The  person  assigned  the  role  of  monitor.  Although  ordinarily 
the  female  head  of  the  household  assumed  the  diary-keeping 
function,  in  some  cases  this  was  performed  by  some  other  member 
of  the  family,  a  neighbor,  a  literate  relative,  or  the  inter- 
viewer herself. 


Inasmuch  as  interviewers  were  told  to  use  a  maximum  effort  in  recruiting 
sample  households  other  forms  of  action  were  used  which  might  be  regarded 
as  special  action,  including  special  letters  to  further  persuade  certain 
families  to  join  the  panel  or  special  calls  to  talk  with  the  husband  to 
overcome  his  suspicion. 
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Table  6  shows  the  results  of  special  action  employed  on  households 
receiving  such  action  as  part  of  the  recruiting  program.  Some  households 
received  more  than  one  form  of  special  recruiting  action,  but  for  purposes 
of  tabulation  a  household  was  classified  on  the  basis  of  the  type  of 
special  action  receiving  most  emphasis.  The  table  suggests  that  special 
action  was  an  important  factor  in  recruiting  sample  households.  Almost 
half  of  the  total  sample  (250  households)  received  one  or  more  forms  of 
special  action  and,  of  households  receiving  such  action,  156,  or  62  percent, 
were  recruited.  This  may  be  contrasted  with  the  record  for  the  270 
households  that  received  no  special  action.  Of  these,  l5h,   or  57  percent, 
were  recruited.  Without  special  action  it  is  doubtful  whether  any  of 
the  households  receiving  such  action  would  have  been  recruited.  With  a 
variable  recruiting  approach  of  the  type  employed  here,  it  is  sometimes 
difficult  to  distinguish  between  special  and  ordinary  recruiting  action. 
Without  doubt  table  6  understates  the  amount  of  special  recruiting 
actually  undertaken.  However,  it  clearly  indicates  that  special  action 
increased  the  over-all  percentage  of  households  recruited. 

Although  special  action  was  a  major  factor  in  inducing  sample  house- 
holds to  join  the  panel,  a  greater  proportion  of  households  receiving  such 
action  discontinued  reporting  (U3  percent)  than  was  the  case  for  the 
"no  special  action"  group  (19  percent).  Thus,  even  though  these  unusual 
measures  are  of  value  in  recruiting  certain  households,  their  effective- 
ness is  somewhat  diminished  when  long-range  cooperation  from  sample 
households  is  desired. 

Table  6.-  Special  recruiting  action  taken  and  results  obtained 


Households 

All 

Recruited 

Reporting 

Type  of  action 

Total 

Per- 
centage 
distri- 
bution 

Total 

:As  a  per- 
centage 
:of  all 
:house- 
: holds 

Total 

:As  a  per- 
centage 
:of  house- 
holds 
: recruited 

Number 

Percent 

Number 

Percent 
57 

Number 
125 

Percent 

• 

270 

52 

151* 

81 

Special  action: 

Method  of  compensation. . 
Change  in  interviewer. . . 
Service  of  bilingual 
interviewer  

15 
93 

36 

106 

3 

18 

7 
20 

57 

ia 

71 

93 
61 

39 
67 

10 
2U 

6 

hi 

71 
U2 

U3 

67 

Speci  al  monitor 

Total  

Grand  Total  

250 
520 

U3 
100 

156 
310 

62 

60 

S7~ 
21? 

57 

68 
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Why  did  certain  households  refuse  to  join  the  panel?  An  analysis  of 
the  reasons  offered  is  presented  in  table  7.  Together,  the  two  cate- 
gories "too  busy"  and  "no  interest"  account  for  80  percent  of  the  reasons 
given  for  refusal  to  join  the  panel.  Reasons  offered  by  refusal  house- 
holds, however,  may  not  in  all  cases  reflect  the  true  motivations  for 
their  failure  to  join  the  panel. 

Having  examined  in  some  detail  the  framework  within  which  recruiting 
operations  were  carried  out,  it  is  appropriate  to  inquire  as  to  its 
adequacy.   But  no  objective  yardstick  exists  for  evaluating  the  adequacy 
of  the  recruiting  efforts  put  forth  on  so  complex  a  survey.  Undoubtedly 
more  cooperation  could  have  been  secured  had  sufficient  compensation  been 
offered  to  overcome  the  objections  of  the  uncooperative  households.   It 
is  also  possible  that  over  a  period  of  time,  the  conditions  preventing 
certain  households  from  joining  the  panel  might  be  removed  and  that  these 
households  could  then  be  recruited.  However,  any  further  efforts  would 
probably  have  yielded  a  relatively  small  return  for  the  resources  expended. 


Table  7.- 


Number  and  percentage  distribution  of  households  not 
joining  the  panel,  by  reason  for  refusal 


Households 

Reason  for  refusal 

Total      :  /efCeffge 
:  distribution 

Too  busy  

No  interest   .  .  . 
Illness   ..... 
Broken  family  .  . 
No  reason  .... 
Other  

Number          Percent 

89             U3 
77             37 
17              8 
7              3 
5               2 
15              7 

Total 

210             100 

TRAINING  AND  MAINTENANCE  OF  SAMPLE  HOUSEHOLDS 


Training  of  Sample  Households 

Once  a  household  had  been  recruited,  the  primary  problem  became  one 
of  training  the  monitor  to  make  accurate  and  complete  entries  in  the  diary. 
For  this  purpose,  a  series  of  training  calls  was  initiated.   (A  training 
call  is  defined  as  a  call  made  for  the  purpose  of  instructing  a  household 
to  keep  the  diary,  to  examine  the  accuracy  of  entries,  and  to  answer 
questions  regarding  problems  arising  in  this  connection.)   It  was  originally 
intended  to  provide  a  minimum  of  two  such  calls  for  every  recruited  house- 
hold. This  ruling  was  la^er  modified,  as  many  monitors  were  able  to  keep 
the  diary  satisfactorily  o  °ter  a  single  training  call.   In  the  event  that 
a  monitor  was  not  adequate!/  trained  after  two  training  calls,  as  many 
further  training  calls  as  necessary  were  provided.  Tables  8  and  9 
summarize  the  extent  of  the  training  effort. 
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About  35  percent  of  the  households  received  one  or  no  training  call, 
whereas  5  or  more  training  calls  were  made  on  approximately  10  percent  of 
the  households.  The  average  per  household  was  2.1*  which  was  the  same 
for  both  reporting  and  dropped  groups  (table  9).  Although  the  results 
are  not  shown  here  in  tabular  form,  the  number  of  training  calls  per 
family  was  unrelated  to  the  number  of  the  call  on  which  the  family  was 
recruited. 

Table  8.-  Number  and  percentage  distribution  of  households  reporting 
and  dropped,  by  number  of  training  calls 


Households 

Number  of 

Reporting 

Dropped 

Total 

training 

: Percentage 

: Percentage 

: Percent age 

calls 

Total 

:  distri- 
:  bution 

■Total 

:  distri- 
:  bution 

Total 

:  distri- 
:  bution 

Number 

Percent 

6 

Number 
13 

Percent 
13 

Number 
26 

Percent 

0 

13 

8 

1 

61 

29 

19 

20 

80 

26 

2 

52 

2U 

2? 

28 

79 

26 

3 

38 

18 

16 

16 

5U 

17 

h 

28 

13 

10 

10 

38 

12 

5 

8 

1* 

8 

8 

16 

5 

6 

6 

3 

2 

2 

8 

3 

7  or  more. 

6 

3 

3 

3 

9 

3 

Total.. 

212 

100 

98 

100 

310 

100 

As  was  done  during  the  recruiting  operations,  a  considerable  part  of 
the  total  training  effort  was  expended  on  not-at-home  training  calls. 
(A  not-at-home  training  call  is  defined  to  be  one  on  which  the  monitor 
being  trained  was  either  not  at  home  or  was  unwilling  to  discuss  diary- 
keeping  procedures  at  the  time  of  a  scheduled  training  call. )  As  shown  in 
table  9,  greater  effort  was  required  to  contact  the  households  that  dropped. 

Table  9.-  Total  and  average  training  calls  and  not-at-home 
training  calls,  by  households  reporting  and  dropped 


Training  calls 


Not-at-home  calls 


Status  of 

household  :  Total  *A™rage  PJ"  Total  ^Average  per 


household 


: training  call 


Reporting 


Dropped  . 
Total 


:  Number 

Number 

Number 

Number 

i   5ll* 

2.1* 

181* 

0.36 

237 

2.1* 

111* 

.1*8 

:    751 

2.1* 

298 

.1*0 
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Maintenance  of  Sample  Households 

For  199  households,  it  was  necessary  to  supplement  the  training  pro- 
gram with  maintenance  calls.   (A  maintenance  call  is  one  made  primarily 
either  to  keep  a  household  reporting  or  to  induce  a  dropped  household  to 
begin  reporting  againT)  As  might  be  expected,  in  some  instances  the 
distinction  between  a  training  call  and  a  maintenance  call  was  not  pro- 
nounced, although  the  latter  were  initiated  exclusively  with  the  super- 
visory staff. 

Of  the  199  households  that  received  maintenance  calls,  39  were 
singled  out  for  what  was  termed  special  maintenance  action.  The  inter- 
viewer was  obliged  to  make  periodic  visits  to  these  39  households  and  to 
give  direct  assistance,  or  to  make  other  unusual  arrangements  to  help 
monitors  keep  the  diaries;  Thus,  special  maintenance  action  may  be 
defined  as  that  group  of  unusual  efforts  required  to  maintain  sample 
households  on  a  reporting  status.  Illustrations  of  special  maintenance 
actions  are:  Giving  direct  assistance  in  keeping  the  diary,  making 
arrangements  for  the  diary  to  be  kept  by  a  neighbor  or  relative,  or  per- 
mitting the  household  to  keep  a  journal-type  rather  than  the  standard 
type  of  diary.  The  significance  of  special  maintenance  calls  in  relation 
to  total  maintenance  calls  is  pointed  up  in  table  10. 

Table  10.-  Number  and  percentage  distribution  of 
maintenance  calls,  by  purpose  of  call 


Purpose  of  call 


To  give  special 

maintenance  help  . 
To  help  person  not  reporting 

or  reporting  poorly  .  . 
To  deliver  special  prize  . 
Other  reasons  1/ t 


Calls 


Total 


: Percentage 
:distribution 


U23 

161* 

10 

172 


Percent 


55 


Total  .  ,.  .  , ;  2/  769 


21 

1 

23 

"Too" 


TJ  This  figure  includes  special  trips  made  to 
obtain  information  on  household  characteristics.   In 
most  cases,  this  purpose  was  combined  with  other 
miscellaneous  reasons  for  making  the  call. 

2/  This  total  includes  55  "maintenance"  calls  on 
house1*  Ids  that  were  never  by  definition  recruited. 
These  households  had  received  a  series  of  training 
calls  since  they  had  agreed  to  join  the  panel.   In 
later  calls  the  interviewers  learned  that  the  diary- 
had  not  been  kept,  but  were  assured  that  diary  keeping 
would  be  resumed  and  that  no  further  training  calls 
were  necessary.  When  no  diaries  were  received,  the  55 
maintenance  calls  were  made.  These  55  calls  are  not 
considered  in  the  remainder  of  this  report. 
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Of  particular  interest  is  the  fact  shown  in  this  table  that  more 
than  half  of  the  maintenance  calls  were  expended  on  the  39  households 
which  received  special  maintenance.   This  tabulation  of  maintenance 
efforts  does  not  take  into  account  such  maintenance  activities  as  tele- 
phone calls  and  letters,  of  which  there  were  more  than  300. 

The  distribution  of  maintenance  calls  among  the  199  households 
receiving  such  calls  is  shown  in  table  11,  in  which  a  further  subdivision 
by  household  status  and  type  of  maintenance  is  made.  Also  included  is 
the  number  of  households  that  received  no  maintenance  calls.  As  the 
recruiting  of  families  extended  over  a  long  period  of  time,  the  distri- 
butions by  number  of  maintenance  calls  are  difficult  to  interpret. 

Although  not  indicated  in  table  11,  the  number  of  maintenance  calls 
received  by  the  households  ranged  from  1  to  53.  The  maximum  number  of 
ordinary  maintenance  calls  made  on  a  household  was  7>  as  contrasted 
with  an  upper  limit  of  53  calls  for  one  household  which  received  special 
maintenance. 

In  table  12,  a  summary  of  the  maintenance-: all  effort  (including 
not-at-home  maintenance  calls)  is  presented  by  type  of  maintenance  within 
household  status.  Thus,  of  a  total  of  '(Ik   maintenance  calls,  about  60 
percent,  or  U23  calls,  were  expended  in  an  effort  to  maintain  39  house- 
holds in  the  panel.   The  average  household  on  special  maintenance 
received  10.8  maintenance  calls  as  opposed  to  an  average  of  1.1  for 
households  not  on  special  maintenance.  Clearly,  households  for  which 
special  maintenance  action  was  necessary  constituted  a  serious  problem 
so  far  as  cost  was  concerned. 

As  proved  to  be  the  case  with  recruiting  and  training  operations, 
the  problem  of  not-at-home  calls  was  serious  in  its  relation  to  the 
maintenance  phase  of  the  project.   (A  not-at-home  maintenance  call  is 
one  made  for  maintenance  purposes  but  unsuccessful  because  the  monitor 
was  not  at  home  or  was  unwilling  to  discuss  diary-keeping  techniques. ) 
It  was  necessary s  then,  to  complete  169  not-at-home  maintenance  calls  in 
order  to  achieve  71U  maintenance  calls.  As  might  be  expected,  when  a 
comparison  with  reporting  households  receiving  the  same  type  of  mainte- 
nance is  made,  the  dropped  households  were  the  worst  offenders  on  a 
per  maintenance  call  basis.   If  differences  between  type  of  maintenance 
are  ignored,  it  may  be  noted  that  reporting  households  on  special 
maintenance  had  the  smallest  number  of  not-at-home  maintenance  calls  on 
a  per  maintenance  call  basis. 

Now  that  the  extent  of  the  efforts  to  train  and  maintain  sample 
households  has  been  examined  in  some  detail,  it  is  appropriate  to  consider 
the  degree  of  success  achieved  by  these  activities.  One  possible  measure 
is  the  extent  to  which  recruited  households  continued  to  report,  although 
this  criterion  is  rather  crude.  Of  the  98  households  that  dropped, 
about  25  percent  reported  for  only  1  or  2  weeks  and  slightly  more  than 
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Table  12.-  Number  of  households  and  maintenance  calls, 
by  status  of  household 


:  Number  of 
i  not- at-home 
s maintenance  calls 


Item 


tMainte-: 
House-  : nance  : 
holds  : calls  : 


Calls 
per 
house- 
hold 


'Total 


:Per  mainte- 
nance call 


Number  Number   Number  Number   Number 


Reporting: 

Special  maintenance  .  • 
Not  on  special 

maintenance  

Dropped: 

Special  maintenance  .  . 
Not  on  special 

maintenance  

31 

\      181 

8 
90 

U03 
182 

20 
109 

13.0 
1.0 

2.5 
lo2 

1*6 

71 

15 
57 

0.11 
.39 

.75 
.52 

Total  reporting  

Total  dropped  

212 

98 

585 
129 

2.3 
1.3 

117 
72 

.20 
.56 

Total  on  special 

maintenance  

Total  not  on  special 

maintenance  

39 
271 

1*23 
291 

10.8 
1.1 

61 
128 

.15 

Grand  total  .... 

310 

nk 

2.3 

189 

.26 

U0  percent  of  these  households  discontinued  within  a  month  after  being 
recruited  (table  13).  An  investigation  of  the  households  which  dropped 
after  reporting  more  than  10  weeks  revealed  that,  in  general,  these  house- 
holds discontinued  because  of  some  external  circumstances,  such  as 
illness  or  moving  away. 

In  some  instances,  it  is  difficult  to  ascertain  why  certain  house- 
holds dropped  from  the  panel.  The  various  reasons  offered  by  the 
households  that  dropped  of  their  own  accord  are  listed  in  table  U4.  In 
a  number  of  cases,  the  reason  for  dropping  was  not  related  to  adequacy  of 
the  training  and  maintenance  programs  (illness,  broken  household). 

Because  of  unsatisfactory  diary  keeping  2U  households  were  dropped 
from  the  panel.  Twenty-one  of  these  households  had  reported  erratically 
over  a  considerable  period  of  time,  and  there  was  no  reason  to  believe 
that  their  reporting  would  improve.  The  other  3  households  could  not 
perform  the  diary-keeping  function  satisfactorily,  and  had  repeatedly 
refused  special  maintenance  assistance. 
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Table  13.-  Number  and  percentage  distribution  of  households 
dropped,  by  number  of  weeks  reporting  before  dropped 


Number  of  weeks 
reporting 
before  dropped 

Households  dropped 

Total     !  HPefCef+a?e 
distribution 

1  .  . 

2  .  . 

3  .  . 
h   .  . 

5  -  8  . 

9-12 
13  -  2U 
25  -  39 

Number         Percent 

Ik                         15 
11           11 

8  8 

9  9 
1U          15 
11           11 
23            23 

8            8 

Total 

98           100 

Table  1U. 


Number  and  percentage  distribution  of  households 
that  dropped,  by  reason  for  dropping 


Households 

dropped 

Reason 

:    Total     j 

Percentage 
distribution 

t         Number 

Percent 

Too  busy  

Illness  

i      32 

t      16 

33 
16 

Not  interested  .  .  . 

11 

11 

Too  much  trouble  .  . 

t                9 

9 

Broken  household  .  . 

i                  2 

2 

Other  reasons  .  .  . 

h 

k 

Total 

Ik 

75 

Company  dropped  (poor 
reporting)  

21 

22 

Gould  not  understand  work; 

refused  maintenance  .  . 

3 

3 

Total 

98 

100 
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COMPARISON  OF  CHARACTERISTICS  OF  HOUSEHOLDS 
BY  STATUS  AND  NUMBER  OF  RECRUITING  CALLS 

As  certain  household  characteristics  and  food-purchasing  behavior  are 
known  to  be  related,  the  personal  characteristics  of  reporting  households 
are  contrasted  with  those  of  dropped  or  refusal  households.   If,  for 
example,  recruited  households  differ  significantly  from  refusal  households 
with  regard  to  such  characteristics  as  age  of  head  of  household  and  size 
of  household,  then  biases  with  respect  to  other  items  are  suspected. 
Unfortunately,  nonsignificant  differences  between  the  two  groups  of  house- 
holds with  respect  to  the  selected  characteristics  do  not  necessarily  mean 
that  the  sample  is  unbiased  with  respect  to  purchases  of  items  included 
in  the  diary. 

The  purpose  of  this  part  of  the  report  is  to  contrast  the  character- 
istics of  certain  groups  of  households  that  exhibited  different  levels  of 
cooperation.  Although  classification  data  were  available  for  all  310 
recruited  households,  detailed  information  on  characteristics  was  available 
for  only  a  subsample  of  121  of  the  210  refusal  households.  Approximate 
tests  of  statistical  significance  of  differences  between  recruited  and 
refusal  households  and  of  differences  between  reporting  and  dropped  house- 
holds have  been  made  (table  15).  The  tests  applied  were  approximate,  as 
the  test  was  based  upon  an  assumption  that  the  sample  was  a  simple  random 
sample  when  actually  stratification  and  clustering  were  involved.  The  test 
used  for  evaluating  the  significance  of  group  differences  was  the  ordinary 
chi-square  test  for  a  k  x  2  table. ir/ 

Judging  from  the  data  given  in  table  15,  households  having  the  follow- 
ing characteristics  were  more  likely  to  be  recruited:   (1)  Of  nonwhite  race; 
(2)  income  under  $6,000.00  per  year;  (3)  many  eaters  per  household; 
(ii)  many  meals  eaten  in  home  per  week;  (5)  younger  household  heads;  (6)  pat- 
ronize no  single  store.  Some  of  these  factors  are  correlated  with  one 
another.   In  particular,  the  number  of  meals  eaten  weekly  in  the  home  is 
correlated  with  the  number  of  eaters  in  the  household;  hence,  if  one  of 
these  two  factors  is  significant,  the  other  one  is  also  likely  to  be  signif- 
icant. Differences  between  the  reporting  and  dropped  households  were  not 
so  striking;  they  differed  significantly  with  respect  only  to  home  ownership 
and  ratio  of  meals  prepared  to  total  possible  meals. 

Another  phase  of  the  general  problem  of  relating  household  character- 
istics to  cooperativeness  is  that  of  determining  whether  the  number  of  the 
call  on  which  a  family  is  recruited  is  related  to  household  characteristics 
(table  16).  The  100  households  recruited  on  calls  after  the  first  tend  to 
differ  from  the  households  recruited  on  first  call,  in  the  same  direction 
that  the  refusal  households  differ  from  the  recruited  households.  This 
would  bear  out  the  importance  of  making  recruiting  calls  beyond  the  first 
but  needs  to  be  interpreted  in  the  light  of  data  given  in  table  2,  which 
showed  that  the  number  of  households  recruited  per  call  did  not  decrease 
materially  with  additional  calls,  and  in  table  3,  which  showed  the  number 
of  households  discontinuing  cooperation  by  call  on  which  they  were  recruited. 


12/  For  a  discussion  of  the  use  of  this  test  see  for  example,  Snedecor, 
G.  W.,  Statistical  Methods.   Iowa  State  College  Press,  19U6.  Ch.  9. 


-  23 


Table  15.-  Characteristics  of  households  by  status-' 


y 


Characteristics 


310 
recruited 
households 


121 
refusal 
households 


212 
reporting 
households 


dropped 
households 


Sampling  area: 

Five  boroughs  .  . 
Remainder  of  area 


Age  of  household  head: 
20  -  3U  years  .  , 
35  -  h9  years  .  . 
50  years  and  over 

Race: 

White  .  

Negro  and  other  . 


Total  household  income: 
Under  $6,000  .  .  . 
$6,000  and  over  .  . 


Employment  of  wife: 
Not  outside  home 
Outside  home  .  . 


Number  of  eaters: 

One 

Two  .  .  .  . 
Three  .  .  .  . 
Four  .... 
Five  or  more 


Ratio  of  meals  prepared 

to  total  possible  meals: 

0.00  -  .59 

.69 

.79 

.89 

1.00 


.60  - 
.70  - 


.90 


Ownership  of  home: 
Own  home  or  are 

furnished  quarters 
Rent  home  


58 
U2 


22 
H2 
30" 


22 

10 


75 

2£ 


6? 
33 


7 

2H 
27 

23 


62 
38 


12 

1:0 


96 

H 


72 

22 


10 
" 

1 

55 

7 


56 
lib 


22 

36 


9 


26 


6? 
33 


9 

2l| 

23 
22 

23 


62 
38 


21 
U2 

37 


!2 


77 
23 


67 

33 


5 
23 
17 
29 
26 


6 

10 

t             6 

7 

6 

10 

I 

11 

13 

15 

i          13 

IF 

23 

26 

t     21 

2H 

52 

38 

55 

39 

33 

39 

38 

21 

67 

61 

32 

79 

2U  - 


Table  15.-  Characteristics  of  households  by  status-  -  Continued 


1/ 


Characteristics 

310 
recruited 
households 

121 
refusal 
households 

212 
reporting 
households 

98 
dropped 
households 

Rent  paid  by  those  who  rents 
0  -  $3U.99  ..... 

$35.00  -  $5U.99 

$55.00  or  more  

Percent 

38 
!     U0 
i           22 

Percent 

31 
3k 
3h 

Percent 

UO 
37 
22 

Percent 

33 
U6 

21 

Total  meals  eaten  ins 
0-21  meals  .  . 
22  -  U2  meals  .  . 
U3  -  63  meals  o  • 
6U  -  8U  meals  .  . 
85  meals  or  more 


Meals  prepared  to  be  taken  out: 

No  meals : 

1-5  meals : 

6-10  meals : 

11  -  15  meals s 

16  -  35  meals s 

Weekly  food  expenditure  for  s 

home  consumptions  s 

0  -  $15.00 : 

$16.00  -  $30.00 : 

$31.00  -  $U5.00  : 

$U6.00  -$100.00 : 


Store  usually  patronized s 
No  single  one  .  .  .  . 
An  independent  •  .  . 
A  chain  ....... 


Education  of  heads 

0-8  years  .  . 

9-12  years  .  . 

13  years  or  more 


2S 

m 
2T 


58 

28 


3  8 
55 

22 

5 


36 
21 

in 


37 
UO 
23 


12 

32 

:t 

?? 


25 

U 
k 


16 

63 

19 

2 


19 
35 
UF 


39 

39 
23 


9 

2U 
25 

20 
22 


57 

27 
9 
5 
2 


18 

56 

19 

6 


3U 

20 
U6 


35 
Ul 

23 


U 
28 
26 
21 
21 


62 

30 

6 

1 

1 


17 

52 
29 
2 


39 

2U 
38 


Ul 
38 
21 


1/  The  entries  are  underscored  for  those  characteristics  having  a  value 
of  chi-square  which  is  beyond  the  5-percent  level  of  significance. 
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Table  16.-  Characteristics  of  households  by  recruiting  call- 


1/ 


Characteristics 


210  households 
recruited 
on  first 
recruiting 
call 


100  households 
recruited 
on  a  later 
recruiting 
call 


Sampling  area: 

Five  boroughs  .  . 
Remainder  of  area 


Age  of  household  head: 
20  -  3k  years  .  . 
35  -  k9  years  .  . 
50  years  and  over 


Race: 


White   

Negro  and  other 


Total  household  income: 
Under  $6,000.00   .  . 
$6,000.00  and  over  . 


Employment  of  wife: 
Not  outside  home 
Outside  home  •  • 


Percent 


S3 
H7 


23 
U3 
35 


12 


76 
25 


67 
33 


Percent 


32 


19 
U2 

ko 


9k 
6 


75 

25 


65 
3k 


Number  of  eaters: 

One 

Two 

Three  •  •  •  • 
Four  .  .  .  . 
Five  or  more  • 


6 

22 

2T 
23 

7E 


10 

27 

20 

■< 


Ratio  of  meals  prepared  to 
total  possible  meals: 


0o00 
.60 
.70 
.30 
.90 


.59 
.69 
.79 
.89 
1.00 


h 

I 
1? 
22 
^6 


12 

T 

17 

»I2 


Ownership  of  home: 

Own  home  or  live  in 

furnished  quarters 
Rent  home   


37 
o3 


-'3 
77 


26 


Table  16.-  Characteristics  of  households  by  recruiting  call-  -  Continued 


y 


Characteristics 


210  households 
recruited 
on  first 
recruiting 
call 


100  households 
recruited 
on  a  later 
recruiting 
call 


Percent 


Percent 


Rent  oaid  by  those  who  rents 

0  -  &3U.99  ...... 

$35.00  -  &U.99   ...... 

$55.00  or  more  ...... 

Total  meals  eaten  at  homes 

0-21  meals  ....... 

22  -  U2  meals  ....... 

U3  -  63  meals  

6U  -  3U  meals  .  .  

85  meals  or  more  

Meals  prepared  to  be  taken  out: 

No  meals  o  

1-5  meals  

6-10  meals  

11  -  15  meals  

16  -  35  meals  ....... 

Weekly  food  expenditure: 

0  -  &15.00  

316.00  -  #30.00  

$31.00  -  IU5.00  

#U6.C0   -$100.00 

Store  usually  patronized: 

No  single  one  

An  independent  

A  chain  ....  

Education  of  head: 

0  -  8  years  

9-12  years  

12  years  or  more  


3^ 

a 
21 


6 

7t> 
10 
2? 


Six 

30 

9 

5 

2 


1G 
bh 
22 

5 


3o 
19 


37 
39 
2U 


36 
39 

2k 


11 

32 

15 

9 


66 

2U 

7 

1 

1 


17 

57 

22 

k 


3k 
2il 

la 


38 

19 


1/  The  entries  are  underscored  for  those  characteristics  having  a  value 
of  chi-square  beyond  the  5-percent  level  of  significance. 
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DESCRIPTION  AND  EVALUATION  OF  SUBSTITUTION  PROCEDURES  EMPLOYED 

With  respect  to  a  sample  as  such  there  are  two  common  practices  for 
treating  the  noncooperation  or  nonresponse  problem.  One  is  simply  to 
increase  the  size  of  the  sample  initially  to  allow  for  noncooperation, 
and  the  second  is  to  select  substitutes.   In  the  latter  case,  attempts 
are  sometimes  made  to  select  substitutes  that  match  the  noncooperators 
with  respect  to  certain  characteristics.  This  was  the  approach  followed 
in  the  present  study.  Some  worth-while  advantages  can  be  claimed  for 
such  a  substitution  procedure,  particularly  for  the  type  of  operation 
described  in  this  report,  but  it  must  be  recognized  that,  technically, 
substitution  is  not  a  satisfactory  solution  to  the  noncooperation  problem. 
In  fact,  the  members  of  the  field  staff  were  strongly  impressed  with  xhe 
need  for  trying  to  gain  the  cooperation  of  the  households  specified  for 
the  sample  and  were  not  told  that  eventually  substitutions  would  be 
allowed.  Incidentally,  the  decision  to  select  substitutes  was  a  blow  to 
the  field  staff  particularly  because  of  the  stress  that  had  been  placed 
on  the  importance  of  recruiting  the  originally  selected  households.   Some- 
what less  stress  on  this  point  probably  would  have  prevented  a  break  in 
staff  morale.  However,  if  interviewers  had  been  told  in  advance  that 
substitutions  would  be  permitted  they  might  have  relaxed  their  efforts 
to  recruit  some  of  the  noncooperative  households. 

In  developing  the  plans  for  selection  of  substitutes  the  general  idea 
was  to  discover  possibilities  of  finding  substitutes  in  the  same  block 
as  the  noncooperators  and  with  similar  food-purchasing  patterns.  Hence, 
the  effort  to  find  substitutes  was  greater  than  would  ordinarily  be 
justified  for  practical  purposes.  As  funds  were  not  available  to  recruit 
a  substitute  for  each  noncooperative  household,  it  was  decided  to  try  to 
recruit  substitutes  for  a  subsample  of  the  noncooperators.  It  was  decided 
to  select  162  substitutes  for  the  3 2lii=y  noncooperative  families.  In 
order  to  select  the  162  noncooperators  for  which  substitution  was  to  be 
effected,  the  32li  noncooperators  were  paired  geographically  and  one  house- 
hold in  each  pair  was  selected  at  random. 

Having  identified  the  162  households  for  which  substitutions  were  to 
be  made,  the  next  step  was  to  obtain  information  as  to  the  food-purchasing 
behavior  of  the  noncooperative  households.  These  households  were  queried 
to  see  whether  they  had  ever  bought  certain  general  types  of  food  products 
such  as  canned  vegetables,  frozen  fruits,  prepared  baking  mixes,  and 
frozen  concentrated  fruit  juice.  In  addition,  questions  regarding  meals 
prepared  in  the  home  were  asked  in  terms  of  individual  household  members, 
so  that  a  summary  figure  for  the  entire  household  could  be  obtained. 
Thus,  information  regarding  both  qualitative  and  quantitative  aspects  of 
food-purchasing  behavior  was  obtained  from  the  noncooperative  households. 


13/  The  noncooperative  group  consisted  of  210  refusals,  100  dropped  (two 
of  these  were  later  re-recruited)  and  Ik   households  which  had  been  called 
on  but  never  contacted. 


-  28  - 

For  each  of  the  162  noncooperative  households  designated,  5  possible 
substitute  households  were  selected.  The  2  households  residing  in  the 
2  dwelling  units  on  either  side  of  the  noncooperator  were  selected,  as 
well  as  3  households  chosen  at  random  from  among  those  residing  on  the 
same  block  as  the  noncooperator.  The  same  information  on  food  purchasing 
was  obtained  from  the  potential  substitute  households  as  was  elicited 
from  the  subsample  of  noncooperators.  In  addition,  information  on  family 
characteristics,  similar  to  that  obtained  for  households  in  the  original 
sample,  was  recorded. 

Considerable  difficulty  was  experienced  in  attempting  to  obtain  the 
desired  information  on  food  purchasing.  Both  the  original  162  noncooper- 
ators and  the  810  potential  substitutes  displayed  considerable  resistance 
when  efforts  were  made  to  elicit  the  desired  information.  In  only  11  cases 
did  the  interviewer  succeed  in  getting  information  from  all  6  households 
(original  noncooperator  and  five  possible  substitutes)  resident  on  a 
given  block.  The  original  noncooperators  were  particularly  resistant; 
in  U8  cases  a  male  interviewer  was  sent  to  obtain  a  minimum  of  information 
from  neighbors  of  the  noncooperator.  The  extent  to  which  interviewers 
were  successful  in  obtaining  the  desired  information  on  food  purchasing 
is  indicated  in  the  following  tabulation: 

Number  of  households  in  a  block 

from  which  information  was  obtained      Frequency 

l 5 

2 11 

3 1*2 

k 63 

5 30 

6 11 

Each  potential  substitute  was  ranked,  as  indicated  below,  according 
to  similarity  to  the  noncooperator,  on  the  basis  of  the  number  of  meals 
eaten  at  home  and  the  number  of  products  commonly  purchased: 


Difference  between  number 
of  meals  eaten  in  home  of 
a  potential  substitute 
and  the  noncooperator 


Difference 


Rank 


0  -  9 5 

10-19 h 

20-29 3 

30-39 2 

UO  or  more 1 


Number  of  products 
commonly  purchased  by 
both  noncooperator  and 
potential  substitute 


Number 

Rank 

25  or  more  . 
20  -  2k    •    •    . 
15-19     •    • 
10  -  lli     .    . 
0  -     9     •   • 

.        5 

.        h 

•        3 

2 

1 
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The  two  ranks  for  each  potential  substitute  were  added  together  and 
the  sum  of  these  two  ranks  was  used  as  an  over-all  rating  of  similarity 
for  purposes  of  designating  the  substitutes  as  to  first  choice,  second 
choice,  etc.  This  gave  the  order  in  which  the  potential  substitutes  were 
to  be  contacted.  When  two  households  had  the  same  over-all  rank  the  tie 
was  broken  by  using  information  on  family  characteristics,,  Information 
on  family  characteristics  was  also  used  for  assigning  ranks  to  those 
households  for  which  adequate  information  on  purchases  was  not  available. 

Of  the  162  substitutes  sought,  only  13k   had  been  successfully 
recruited  and  were  reporting  as  of  July  15,  1951  -  6  months  after  the 
recruiting  of  substitutes  was  started.  Of  the  13i;  substitutes  successfully 
reporting  by  that  date,  a  total  of  93   had  been  obtained  through  the  use 
of  the  substitution  plan  described  above.   In  the  remaining  kl   cases,  the 
pool  of  potential  substitutes  had  been  exhausted,  and  it  was  necessary  to 
obtain  substitutes  from  households  on  the  block  other  than  those  designated 
by  the  original  substitution  scheme.  In  these  latter  cases,  interviewers 
were  instructed  to  begin  with  the  household  located  next  to  the  fifth  or 
last  potential  substitute  and  proceed  around  the  block  clockwise  until  a 
household  was  recruited.  The  number  of  substitutes  actually  recruited  by 
order  of  choice  was: 

Order  of  choice      Number  of  households 


First hk 

Second 35 

Third 10 

Fourth  3 

Fifth 1 

Not  designated Ul 

Total IJU 

Some  indication  of  the  magnitude  of  the  task  of  obtaining  cooperating 
substitutes  is  provided  by  the  data  on  number  of  households  contacted  for 
substitution.  To  obtain  the  13U  recruited  substitutes,  it  was  necessary 
to  contact  503  substitute  households.   Of  the  substitute  households 
contacted,  195  refused  outright  and  1'fh   joined  but  later  dropped  out  of 
the  panel.  Furthermore,  although  a  limit  of  2  recruiting  calls  and  3 
not-at-home  calls  were  made  on  the  potential  substitutes,  a  total  of  913 
not-at-home  calls  were  made  on  the  503  potential  substitutes  actually 
contacted. 

As  compared  to  the  520  households  contacted  in  the  original  sample, 
about  the  same  percentage  of  the  503  substitute  households  agreed  to 
cooperate  but  the  drop-out  rate  was  considerably  higher.   The  contacts 
with  households  in  the  recruiting  of  substitutes  was  less  deliberate  than 
was  the  case  with  the  original  sample.  Previous  experience  has  indicated 
that  strong  persuasive  efforts  to  hurry  a  decision  to  cooperate  will  cause 
as  high  or  perhaps  a  higher  proportion  to  agree  to  cooperate  than  more 
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deliberate  efforts  but  that  the  rate  of  drop  out  of  those  hurriedly- 
persuaded  will  probably  be  higher,  thus  resulting  in  the  end  in  a  lower 
proportion  actually  reporting.  However,  this  result  could  be  accounted 
for  by  the  fact  that  substitutes  were  selected  from  blocks  in  which  the 
cooperation  rates  are  lower  than  average. 

To  the  degree  that  substitution  was  successful,  it  may  be  expected 
that  the  food-purchasing  habits  of  the  substitute  households  will  be 
similar  to  those  of  the  noncooperative  households.  However,  since  no 
information  is  available  on  the  actual  purchases  of  various  items  by  the 
noncooperative  households,  no  comparisons  can  be  drawn  between  the  two 
groups  in  this  regard.  Nevertheless,  certain  characteristics  of  the 
group  of  reporting  substitutes  can  be  compared  with  the  group  of  original 
noncooperators  and  the  first-choice  substitutes  (table  17). 

In  the  preceding  section,  use  was  made  of  the  chi-square  test  to 
determine  whether  significant  differences  existed  in  the  composition  of 
recruited  households  as  contrasted  with  refusal  households.   Unfortu- 
nately, this  test  cannot  be  used  in  the  present  context  and  reliance  must 
of  necessity  be  placed  on  inspection  of  the  data.   In  general,  reports 
of  the  substitute  cooperators  appear  to  be  more  nearly  like  those  of  the 
original  cooperators  who  are  still  reporting  than  the  reports  of  the 
refusals. 

Thus,  although  considerable  effort  was  expended  on  the  task  of 
getting  cooperating  substitutes,  certain  biases  apparently  remained  in 
the  sample  of  cooperators  actually  recruited.  The  magnitudes  of  any 
biases  in  terms  of  actual  food  purchases  cannot  be  accurately  assessed 
from  available  information. 

It  is  clear  that  because  of  the  problems  previously  cited,  the  plan 
for  enlisting  substitutes  originally  designed  was  too  complex.  The 
necessary  information  on  food  habits  was  extremely  difficult  to  obtain. 
This  coupled  with  the  general  problem  of  getting  cooperation,  tended  to 
diminish  the  value  of  the  plan  as  a  practical  technique  for  obtaining 
substitute  households.  In  future  operations  of  this  kind,  when  substitu- 
tion is  necessary,  the  plan  selected  should  be  much  simpler,  and  should 
result  in  getting  substitutes  more  nearly  like  the  refusals  than  the 
households  already  cooperating. 
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Table  17.-  Characteristics  of  original  noncooperators, 
first-choice  substitutes  and  reporting  substitutes 


Characteristics 


162 

original 

noncooperators 


13k         ; 

first  choice ] 
substitutes 


131* 
reporting 
substitutes 


Age  of  household  head: 

'   20  -  3U  

15  -  k9   

SO   years  and  over  . 


Race: 


White  

Negro  and  other 


Total  household  income: 
Under  $6,000.00  .  . 
$6,000.00  and  over 

Employment  of  wife: 
Not  outside  home 
Outside  home  .  .  . 


Number  of  eaters: 

One 

Two 

Three  .... 
Four  .... 
Five  or  more 


Ratio  of  meals  prepared  to 

total  possible  meals: 

0o00  -  0,59 

.60  -   .69 

.70  -   .79 


.90 


1.00 


Ownership  of  home: 

Own  home  or  live  in 

furnished  quarters 
Rent  home  


Percent 


13 
U3 


% 
6 


71 

29 


70 

30 


9 
29 
28 
20 
Ik 


9 
12 
15 
27 

37 


37 

63 


Percent 


2k 

37 
39 


7 


■n 

?! 


6y 
31 


h 
37 
2? 
1.8 
12 


9 

5 

10 

25 

51 


38 

6? 


29 
111 

29 


95 
5 


76 
2k 


73 

2? 


3 
25 
26 
27 
18 


3 
6 

10 
20 
61 


35 
65 
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Table  17.-  Characteristics  of  original  noncooperators,  first-choice 
substitutes  and  resorting  substitutes  -  Continued 


Characteristics 


162 

;      uu 

:     1324 

original 

first  choice 

\   reporting 

noncooperators 

[substitutes 

[substitutes 

Percent 

Percent 

Percent 

32 

29 

32 

36 

k3 

U2 

33 

28 

26 

10 

7 

2 

32 

37 

26 

28 

28 

29 

22 

20 

27 

9 

7 

15 

70 

67 

59 

2k 

21* 

25 

6 

7 

11 

- 

1 

a 

12 

17 

9 

eh 

58 

58 

20 

20 

27 

k 

5 

6 

20 

10 

37 

23 

28 

1*0 

57 

62 

UO 

3U 

33 

U2 

U2 

23 

21* 

21* 

Rent  paid  by  those  who  rent: 
0  -  $3k.99 
$35.00  -  $51*.  99 
$55.00  or  more 


Total  meals  eaten  in 
0  -  21  .  . 
22  -  1*2  .  . 
Ii3  -  63  .  . 
61*  -  81*  .  . 
85  meals  or  more 


Meals  prepared  to  be  taken  out 
No  meals  ...  

1  -  5 

6-10 

11  or  more  


Weekly  food  expenditure: 

0  -  $15.00  . 

$16.00  -  $30.00  . 

$31.00  -  $1*5.00  . 

$1*6.00  -  $100.00  . 


Store  usually  patronized 
No  single  one  ... 
An  independent  .  . 
A  chain  


Education  of  head: 

0-8  years  .  . 

9-12  years  .  . 

13  years  or  more 
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SUMMARY  AND  CONCLUSIONS 

This  report  has  presented  the  results  of  an  intensive  effort  to 
secure  and  maintain  the  cooperation  of  a  probability  sample  of  5U6  house- 
holds in  the  New  York  metropolitan  area  for  a  continuous  consumer  panel. 
The  primary  purpose  of  the  project  was  to  examine  the  problems  associated 
with  the  use  of  a  probability  design  for  establishing  such  a  panel.  The 
goal  of  the  project  was  to  obtain  insofar  as  possible  complete  and 
accurate  information  on  certain  phases  of  the  food-purchasing  behavior 
of  the  households  specified  by  the  sampling  design. 

A  field  force  of  11  women  interviewers  was  hired  to  recruit,  train, 
and  maintain  the  sample  households  as  reporting  members  of  the  consumer 
panel.   In  an  attempt  to  obtain  maximum  cooperation  from  the  sample 
households,  the  general  policy  was  to  make  as  many  recruiting,  training, 
and  maintenance  calls  as  were  necessary  to  establish  rather  definitely 
that  further  effort  would  not  lead  to  the  cooperation  of  those  contacted 
or  to  better  reporting  by  those  cooperating.  For  the  most  part,  the 
information  sought  was  to  be  transmitted  by  mail  in  the  form  of  weekly 
diaries  prepared  by  household  monitors,  although  in  some  cases  a  more 
direct  form  of  communication  was  employed.   For  successful  execution  of 
their  role  in  the  project,  the  sample  households  were  remunerated  in 
points  equivalent  to  about  &2.15  a  month. 

As  part  of  the  general  effort  to  obtain  maximum  cooperation,  several 
forms  of  special  recruiting  and  maintenance  action  were  undertaken. 
Among  the  more  important  of  these  were:   (1)  Use  of  a  variable  recruiting 
approach,  modified  to  fit  the  particular  situation  at  hand;  (2)  alterna- 
tion of  interviewers,  when  this  appeared  necessary;  (3)  use  of  bilingual 
interviewers,  for  non-English  speaking  households;  (h)   assignment  of 
special  monitors,  to  households  where  the  female  head  was  unable  to  assume 
the  diary -keeping  function;  (5)  modification  of  the  compensation  scheme, 
when  this  appeared  desirable. 

In  addition,  severaj.  types  of  special  action  were  carried  out  in  an 
attempt  to  maintain  recruited  households  on  an  active  reporting  basis. 
In  particular,  a  number  of  households  were  visited  periodically  by 
interviewers  and  given  direct  assistance  in  keeping  the  diaries.  Also, 
special  provisions  were  made  for  reporting  by  certain  households  with  the 
assistance  of  a  neighbor  or  a  relative. 

The  carrying  out  of  these  procedures  was  expensive.  As  an  indication 
of  the  total  magnitude  of  the  field  effort,  a  summary  of  all  calls  made 
on  the  original  sample  of  cooperators  by  type  of  call  and  status  of 
households  is  shown  in  table  18.  This  table  discloses  that  not-at-home 
calls  of  all  types  accounted  for  most  of  the  calls  made.   Also,  if  not- 
at-home  calls  are  segregated  by  type,  it  will  be  seen  that  recruiting 
not-at-home  calls  accounted  for  the  greater  part  of  the  not-at-home  calls. 
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Table  18.-  Summary  of  type  of  calls   by  status  of  household 


y 


Type  of  call 

Households 

Reporting 

Dropped 

Refused 

Total 

Number 

Percent 

Number 

Percent 

Number  Percent 

Number 

Percent 

Recruiting  .    . 

Training  .    .    . 

Maintenance : 
Special     .    . 
Regular     .    . 

289 
51U 

U03 
182 

15 
27 

21 
10 

165 
237 

20 

109 

20 

29 

2 
13 

386 
93 

55 

28 
7 

k 

8U0 
8UU 

U23 
3h6 

20 
21 

10 
9 

Total     .    . 

585 

31 

129 

15 

55 

h 

769 

19 

Not-at-home: 
Recruiting   . 
Training   .    . 
Maintenance. 

201 
18U 

-       117 

11 

10 

6 

11U 

llh 

72 

1U 

ia 

8 

729 
58 
37 

k 

3 

i,ouu 

356 

226 

26 
9 
5 

Total     .    . 

i        502 

27 

300 

36 

82U 

61 

1,626^ 

ko 

Total   all  calls 

1,890 

100 

831 

100 

1,358 

100 

U.079 

100 

1/  This  summary  of  calls  refers  only  to  those  made  in  connection  with  the 
original  sample.  No  calls  expended  on  recruiting  of  substitutes  are  included. 

As  many  as  6  recruiting  calls  were  made  on  some  households.  However, 
the  making  of  more  than  one  recruiting  call  is  regarded  as  worth  while  in 
view  of  (1)  the  number  of  households  recruited  per  recruiting  call  for  calls 
after  the  first  (table  2)j  (2)  the  rate  of  dropping  from  the  panel  by  call 
on  which  recruited  (table  3) J  (3)  the  number  of  training  calls  per  household 
by  call  on  which  recruited;  and  (U)  the  fact  that  households  offering  the 
greatest  resistance  to  cooperation  differ  from  the  more  cooperative  households. 

Although  60  percent  of  the  households  agreed  to  cooperate,  only  35 
percent  were  reporting  in  February  1951,  which  was  about  1  year  after  the 
recruiting  started.  Thus,  obtaining  and  maintaining  cooperation  proved  diffi- 
cult throughout  the  project.  Although  the  problem  of  "missing  data"  is 
virtually  always  important  in  sampling  from  human  populations,  it  was  found  to 
be  particularly  acute  in  this  study.  The  chief  cause  for  this  is  the  fact 
that  sustained  cooperation  was  required  of  the  households.  In  addition,  it 
is  likely  that  the  complexity  of  the  task  required  of  the  respondents  contrib- 
uted to  the  severity  of  the  problem. 

Although  it  is  comparatively  easy  to  cite  reasons  for  lack  of  cooperation, 
it  is  quite  another  matter  to  assess  the  importance  of  this  factor  or  to 
suggest  ways  of  solving  the  problem.  If  cooperation  were  independent  of  food- 
purchasing  behavior,  the  problem  would  be  of  little  consequence.  However, 
this  study  indicates  that  certain  household  characteristics,  known  to  be 
related  to  food  purchases,  are  associated  with  degree  of  panel  cooperation. 
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Because  of  these  known  biases,  it  is  not  possible  to  regard  the  sample  of 
reporting  households  as  a  random  sample  from  the  population  of  interest. 
Moreover,  as  the  food  purchases  of  the  noncooperative  households  is  not 
known,  no  estimate  of  the  magnitude  of  the  resultant  bias  can  be  made. 

One  way  of  attacking  the  problem  of  gaining  a  higher  rate  of  coopera- 
tion is  to  increase  the  remuneration  offered  the  sample  households.  There 
is  no  doubt  that  this  would  have  led  to  a  higher  rate  of  cooperation  than 
was  in  fact  obtained.  However,  when  queried  as  to  the  probable  influence 
of  such  action,  the  interviewers  thought  that  the  compensation  would  have 
to  be  increased  200  to  300  percent  to  effect  an  appreciable  increase  in 
cooperation.  As  the  present  project  was  carried  out  within  a  given  frame- 
work of  economic  resources,  any  increase  in  compensation  of  this  order 
was  not  possible. 

Another  way  of  coping  with  the  problem  of  noncooperation  is  to  recruit 
a  subsample  of  noncooperators  to  report  either  continuously  or  for  selected 
weeks  or  months  during  the  year.  Experience  indicates  that  the  purchasing 
patterns  of  cooperators  are  changed  during  their  first  few  weeks  of 
reporting  but  that  after  that  the  purchasing  pattern  evidently  returns  to 
about  "normal."  Hence,  getting  the  cooperation  of  a  family  to  keep  diaries 
intermittently  is  of  doubtful  value.  As  an  alternative,  one  might  contact 
the  noncooperating  households  (or  a  subsample  of  them)  periodically  and 
ask  for  information  on  purchases  of  the  previous  week.  This  might  provide 
useful  information  upon  which  to  make  adjustments  for  noncooperation. 
However,  after  consultation  with  the  interviewers  and  inspection  of  the 
history  of  contacts  with  the  noncooperating  households,  these  possibilities 
were  ruled  out  as  impracticable,  but  might  have  been  worth  a  trial  before 
repeated  recruiting  efforts  tended  to  weaken  rapport  with  the  households. 

A  third  method  of  dealing  with  the  noncooperation  problem  is  the  one 
used  in  this  study.  It  was  to  select  substitutes  for  noncooperating 
households,  although  statistically,  substitution  is  not  a  desirable  solu- 
tion. An  attempt  was  made  to  select  a  substitute  that  would  match  the 
noncooperating  households  insofar  as  possible  with  respect  to  number  of 
meals  eaten  in  the  home  and  the  kinds  of  foods  bought.  Five  households 
resident  on  the  same  block  as  the  original  noncooperator  were  contacted, 
and  certain  information  was  obtained  from  them.  These  households  were 
ranked  as  first  choice,  second  choice,  etc.  and  contacted  in  the  order 
rated  until  one  of  the  households  agreed  to  cooperate.   If  none  of  the  des- 
ignated substitutes  was  successfully  recruited,  efforts  were  made  to  secure 
the  cooperation  of  other  households  resident  on  the  same  block  as  the 
noncooperator. 

Taking  everything  into  account,  the  substitution  plan  was  much  too 
complex  for  practical  use.  However,  the  substitution  work  done  on  this 
project  was  primarily  exploratory  and,  as  such,  it  was  anticipated  that 
difficulties  would  arise.  No  precedent  existed  for  the  use  of  such  a 
technique  in  connection  with  an  operation  of  this  type. 
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Even  though  considerable  effort  was  made  to  find  a  substitute  to 
match  each  noncooperator,  the  characteristics  of  the  substitutes 
recruited  tended  to  be  more  like  the  characteristics  of  cooperating  house- 
holds than  those  of  the  refusals.  A  complicated  substitution  scheme  is 
of  doubtful  value;  but,  if  matching  is  attempted,  a  simple  device  of 
matching  on  the  basis  of,  for  example,  family  size  within  the  same  block- 
is  probably  as  good  as  a  more  complicated  procedure.  Perhaps  the  most 
that  can  reasonably  be  expected  is  to  devise  a  method  of  maintaining  the 
sample  size  at  a  fixed  level  and  of  keeping  the  sample  properly  allocated 
geographically  by  selecting  substitutes  within  the  same  blocks.  The 
latter  will  have  some  tendency  to  keep  proportionate  representation  by 
income  areas,  race,  and  other  factors  associated  with  geographic  location. 
By  dividing  the  sample  data  into  groups  on  the  basis  of  size  of  household 
or  one  or  more  of  several  factors  and  expanding  the  sample  data  for  each 
group  by  known  total  numbers  of  households  in  the  groups,  the  question  of 
bias  due  to  noncooperation  becomes  one  of  difference  between  cooperative 
and  uncooperative  households,  for  example,  within  size  of  household  groups 
and  geographic  areas. 

The  two  most  troublesome  problems  were  high  costs  and  missing  data. 
An  important  question  to  be  considered  was:  Is  a  probability  design  of 
value  in  establishing  a  continuous  consumer  panel?  A  casual  inspection 
of  the  findings  might  suggest  that  the  value  of  the  design  was  nullified 
because  of  the  failure  to  attain  a  high  rate  of  cooperation  from  sample 
households  but  that  some  advantages  might  be  claimed.   It  is  helpful  to 
think  of  the  population  of  households  as  made  up  of  two  subpopulations — 
cooperators  and  noncooperators.  A  probability  sample  of  the  cooperator 
subpopulation  can  be  obtained  and  "statistically"  unbiased  estimates  and 
sampling  errors  relating  to  it  can  be  computed.  The  problem  is  then  one 
of  trying  to  get  sufficient  information  on  the  subpopulation  of  noncooper- 
ators to  provide  a  sounder  basis  for  making  estimates  for  the  total 
population  than  those  provided  by  information  from  the  cooperators  only. 

Despite  the  problem  of  noncooperation  it  is  believed  that  a  better 
way  to  obtain  data  than  quota  sampling  involving  nonrandom  selection,  is 
to: 

(1)  Begin  with  a  probability  sample  of  households  and  attempt  to 
recruit  all  households  in  the  sample  but  limit  the  number  of 
recruiting  calls  to  three  or  four. 

(2)  Obtain  from  cooperators  a  limited  amount  of  information  on 
their  characteristics  and  food  purchases  as  an  aid  to  evaluating 
biases  due  to  noncooperation.  One  possibility  would  be  to  ask  at 
the  time  of  the  first  recruiting  call  (perhaps  before  discussing 
panel  membership)  for  information  on  last  week's  actual  purchases 
of  certain  foods  regardless  of  the  household's  attitude  toward 
joining  the  panel.  This  would  give  a  method  of  comparing  the 
eventual  cooperators  and  noncooperators  on  actual  purchases,  as 
only  a  small  percentage  of  all  households  would  refuse  to  give 
such  information.  This  scheme  assumes  that  recall  error  is  about 
the  same  for  both  groups. 
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(3)  Select  a  substitute  for  each  noncooperator  from  within  the 
same  block  by  proceeding  around  the  block  systematically  to  keep 
the  selection  objective.  Further  work  is  needed  to  determine 
the  practicability  of  trying  to  select,  for  example,  a  family  of 
approximately  the  same  size. 

Compared  to  quota  sampling  the  above  procedure  is  objective  in  its 
approach,  and  provides  a  better  basis  for  judging  the  sample  and  also  a 
better  basis  for  building  in  improvements  from  time  to  time.   Cost 
comparisons  have  little  or  no  meaning  unless  the  alternatives  being 
compared  are  specified  in  detail  as  to  procedure.  Assuming  comparable 
scatter  of  the  sample,  comparable  effort  to  recruit  a  household  contacted, 
and  comparable  training  of  monitors,  there  will  probably  be  little 
difference  in  cost  between  quota  and  probability  sample  designs. 
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Appendix  A 

ITING  CALL  R 

Name  of  family Family  No. 

Address Interviewer_ 

City State  Date  

Phone 


1.  Full  name  of  person  interviewed 
Position  in  household 


2.  Is  this  the  individual  who  plans  the  meals?   Yes No 

3.  Is  this  the  individual  who  makes  a  majority  of  the  food  and  drug 

purchases?    Yes No 

ii.  If  answer  to  either  question  #2  or  #3  is  'no',  give  details, 

including  name  and  position  in  family  of  person  who  does. 

5.  Whom  do  you  recommend  as  the  potential  family  monitor? 

Name 

Address  (if  other  than  family)         

Position 

6.  Was  anyone  other  than  respondent  present  at  interview? 
Yes No If  'yes',  who? 

What  was  the  effect  on  you  and  the  respondent? 

7.  Grade  the  respondent's  reaction  or  interest  in: 

Favorable  No  Reaction  Unfavorable 
Letters  of  introduction 
Importance  of  consumer  panel  work 

Pay  (points)  for  diary  keeping      

Quality  and  diversity  of  premiums   

Diary  form 
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RECRUITING  CALL  REPORT^ -  Continued 


B  If  family  agreed  to  become  a  member,  fill  out  the  following  section: 

1.  Check  type  of  diary  control.   Mail  Pick  up 

2.  If  pick  up,  write  date  of  next  appointment  


3.  What  frequency  of  contact  do  you  recommend  with  family? 

U.  If  family  monitor  is  other  than  person  recommended  in  section  A, 
fill  in  name  and  position,  and  explain  circumstances. 


Address  (if  other  than  family) 
Position 


5.  Has  any  family  member  expressed  any  doubts  about  joining  panel? 
Yes No If  'yes',  describe  in  detail 

6.  Who  receives  the  benefit  of  the  points? 


C  If  family  does  not  agree  to  become  a  member,  fill  out  the  following 
section: 

1.  What  are  the  reasons  for  not  joining? 

2.  What  future  action  do  you  recommend? 

3.  Was  future  appointment  made?    Yes No 

Date  

Comments: 


1/  On  the  actual  reporting  form  space  was  allowed  for  answers  to  all 
questions. 
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Appendix  B 
TRAINING  CALL  REPORT 


y 


(Attach  this  form  to  Diary 
or  Diaries  discussed.) 

Name  of  family  Family  No. 

Address Interviewer 

City  State  Date  


1.  Dates  of  diary  discussed.  From  to 


2.  Did  you  have  opportunity  to  go  over  all  errors  in  the  diary  with 
family  monitor?  Yes No .  If  'no1,  explain  circumstances. 

3.  What  was  family's  reaction  to  entry  corrections  and  training 
with  diary? 

U.  Is  monitor  using  immediate  entry  system?   Yes No 

If  'no',  describe  system  used  and  any  action  taken  to  improve  it. 

5.  Are  errors  chiefly  the  result  of:  1.  Carelessness  Yes No 

2.  Misunderstanding 
of  instructions 

3.  Other  reason 

Give  details 

6.  What  questions  and/or  criticism  did  members  of  the  family  have? 
(If  none,  so  indicate)  . 

A.  About  making  entries  in  the  diary  


B.  About  obtaining  cooperation  from  other  members 
of  the  family 


C.  About  the  premiums  (Any  dissatisfaction  with 
rate  of  compensation) 


D.  Any  other  (describe  fully) 
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(Attach  this  form  to  Diary 
or  Diaries  discussed. ) 


Y.  What  is  the  general  attitude  of  the  family  toward  continuing  as  a 
member  of  the  National  Consumer  Panel? 

Excellent Good Fair Poor 

Explain: 

8.  Has  the  attitude  of  the  family  become  stronger ,  weaker__ , 

remained  about  the  same ,  when  compared  to  the  last  call. 

Explain: 

9.  Did  the  respondent  make  any  comments  about  her  weekly  purchases? 

Yes No 

Explain:   (Note  any  remarks  such  as  "I'm  sorry  I  bought  so  little 
last  week,  but  ...",  or  "I  never  knew  widgies  came  in 
two  flavors1*,  or  statements  that  diary  keeping  makes 
shopping  easier,  etc.) 

10.  What  action  do  you  recommend  the  company  take  to  improve  relations 
with  the  family? 

11.  Date  of  next  appointment  . 


1/  On  the  actual  reporting  form  space  was  allowed  for  answers  to  all 
questions. 
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